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(54) Dynannic intra-coded macrobiock refresh interval for video error concealment 



(57) The present invention provides, in one embod- 
iment, a system and method for concealing video errors. 
The system includes a coding engine (402) for process- 
ing each frame of a video signal to generate macrob- 
locks, selecting a refresh Interval based upon network 
communication parameters, and encoding one or more 
of the macroblocl<s as refresh Intra-coded macroblocks 
based upon the selected refresh interval. The refresh 



intra-coded macroblocks are placed into discrete data 
packets for transmission across a network to one or 
more remote devices. Upon receiving other refresh in- 
tra-coded macroblocks from a remote device via the net- 
work, the coding engine (402) decodes the received in- 
tra-coded macroblocks, and places the decoded mac- 
roblocks in a newly decoded video frame to be used as 
reference frame macroblocks for reducing video errors 
associated with prediction drift. 
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Description 

BACKGROUND OF THE INVENTION 

1 . Field of the invention 

[0001 ] The present invention relates generally to vid- 
eo communication, and more particularly to video error 
concealment. 

2. Description of Related Art 

[0002] Video images have become an Increasingly 
important part of global communication. In particular, 
video conferencing and video telephony have a wide 
range of appiications such as desictop and room-based 
conferencing, video over the Internet and over tele- 
phone lines, surveillance and monitoring, telemedicine, 
and computer-based training and education. In each of 
these appiications, video and accompanying audio in- 
formation is transmitted across telecommunication 
links, including telephone lines, ISDN, DSL, and radio 
frequencies. 

[0003] A standard video fonnat used in video confer- 
encing is Common Intemiediate Fonnat (CIF), which is 
part of the International Telecommunications Union 
(ITU) H.261 videoconferencing standard. Additional for- 
mats with resolutions higher and lower than CIF have 
also been established. FIG. 1 Is a table of the resolution 
and bit rate requirements for various video formats un- 
der an assumption that 12 bits are, on average, required 
to represent one pixel. The bit rates (in megabits per 
second, Mbps) shown are for uncompressed color video 
frames. 

[0004] Presently, efficient transmission and reception 
of video signals may require encoding and compression 
of video and accompanying audio data. Video compres- 
sion coding Is a method of encoding digital video data 
such that less memory is required to store the video data 
and a required transmission bandwidth is reduced. Cer- 
tain compression/decompression (CODEC) schemes 
are frequently used to compress video frames to reduce 
required transmission bit rates. Thus, CODEC hardware 
and software allow digital video data to be compressed 
Into a more compact binary format than required by the 
original (i.e., uncompressed) digital video format. 
[0005] Several conventional approaches and stand- 
ards to encoding and compressing source video signals 
exist. Some standards are designed for a particular ap- 
plication such as JPEG (Joint Photographic Experts 
Group) for still Images and H.261 , H.263, MPEG (Mov- 
ing Pictures Experts Group), MPEG-2, and MPEG-4for 
moving images. For moving images, these coding 
standards, typically, use block-based motion-compen- 
sated prediction on 16x16 pixels, commonly referred to 
as macrobiocks. In one embodiment, a macroblock is a 
unit of Information containing four 8x8 blocks of lumi- 
nance data and two corresponding 8x8 blocks of 



chrominance data in accordance with a 4:2:0 sampling 
structure, where the chrominance data Is subsampled 
2:1 in both vertical and horizontal directions. 
[0006] For applications in which audio accompanies 

5 video, as a practicality, audio data also must be com- 
pressed, transmitted, and synchronized along with the 
video data. Synchronization, multiplexing, and protocol 
Issues are covered by standards such as H.320 (ISDN- 
based video conferencing), H.324 (POTS-based video 

10 telephony), and H.323 (LAN or IP-based video confer- 
encing). H.263 (or its predecessor, H.261) provides the 
video coding part of these standards groups. 
[0007] A motion estimation and compensation 
scheme Is one conventional method typically used for 

15 reducing transmission bandwidth requirements for a 
video signal. Because the macroblock is the basic data 
unit, the motion estimation and compensation scheme 
may compare a given macroblock In a current video 
frame with the given macroblock's surrounding area In 

20 previously transmitted reference video frames, and at- 
tempt to find a close data match. If a close data match 
is found, the scheme subtracts the given macroblock in 
the current video frame from a closely matched, offset 
macroblock in a previously transmitted reference video 

25 frame so that only a difference (i.e., residual) and the 
spatial offset needs to be encoded and transmitted. The 
spatial offset is commonly referred to as a motion vector. 
If the motion estimation and compensation process is 
efficient, the remaining residual macroblock should con- 

30 tain a small amount of Information thereby leading to ef- 
ficient compression. 

[0008] Video data may be transmitted over packet 
switched communication networks or on heterogeneous 
communications networks in which one of the endpoints 

35 is associated with a circuit-switched network, and a 
gateway or other packet-switched to circuit switched 
network bridging device is used. When preparing video 
frame information for transmission over a packet 
switched communication network, encoding schemes 

40 transform the video frame information, compressed by 
motion estimation and compensation techniques or oth- 
er compression schemes into data packets for transmis- 
sion across the communication network. Data packets 
are sometimes lost, duplicated, or delayed which can 

45 Introduce errors resulting in video quality degradation. 
[0009] For example. If one or more data packets of a 
previously transmitted reference frame are lost upon 
transmission from a source encoding unit to a target de- 
coding unit, then a mismatch between encoder and de- 

50 coder reference frames typically results. When the en- 
coder and decoder reference frames are not mis- 
matched, a residual computed and transmitted by the 
encoder is decoded and added to a motion compensat- 
ed video frame derived from the decoder's reference 

55 frame. Roughly speaking, in the absence of transmis- 
sion errors, the resulting decoded video frame exactly 
matches the encoder's reference frame. When a refer- 
ence frame mismatch occurs, the sum of the decoded 
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residual and the decoder's motion compensated video 
frame results In a decoded video frame that further dif- 
fers from the encoder's reference frame. Without cor- 
rection, these differences, called prediction drift, in- 
crease until the decoded video becomes unintelligible 
even if subsequent encoded video is received error free. 
[0010] Therefore, there is a need for a system and a 
method to conceal errors caused by data paclcet loss 
and reference frame mismatches, thereby improving 
video quality. 

SUMMARY OF THE INVENTION 

[001 1 ] The present invention reduces video error gen- 
eration, persistence, propagation, and amplification 
caused by reference frame mismatches associated with 
packet loss of video data. In general, the present inven- 
tion provides a system and method that dynamically se- 
lects a refresh Interval based upon networl< communi- 
cation parameters, and intra-codes a specific pattern of 
macroblocks based upon the selected refresh interval. 
Since intra-coded macroblocks are bit expensive, the 
number of macroblocks intra-coded per video frame 
based upon the selected refresh interval Is balanced by 
an increase In data transmission cost. In an exemplary 
embodiment of the invention, a coding engine is provid- 
ed for processing each frame of a video signal to gen- 
erate macroblocks, selecting a refresh interval based 
upon network communication parameters, and encod- 
ing one or more of the macroblocks as refresh Intra-cod- 
ed macroblocks based upon the selected refresh inter- 
val. 

[0012] In a further exemplary embodiment, a video 
signal is transformed into a plurality of macroblocks, 
where each frame of the video signal has an Integer 
number of macroblocks dependent upon a video signal 
format. Then, a refresh interval is selected based upon 
network call rates and packet-loss error rates, and one 
or more of the plurality of macroblocks are intra-coded 
in a specific pattern based upon the selected refresh in- 
terval. These macroblocks are referred to as refresh in- 
tra-coded macroblocks. Macroblocks that are not intra- 
coded as refresh macroblocks are either intra-coded or 
inter-coded based upon any number of selection crite- 
ria. For example, a given macroblock may be inter-cod- 
ed If data content of the given macroblock and data con- 
tent of an offset macroblock from a previous frame are 
similar, and intra-coded otherwise. 
[0013] In another embodiment of the invention, the 
encoded macroblocks are placed into discrete data 
packets for transmission over a network to one or more 
remote devices. In addition, refresh Intra-coded mac- 
roblocks received from a remote device via the network 
are used in lieu of macroblocks motion compensated 
from a previous video frame to reduce prediction drift. 
[0014] In another embodiment of the invention, the re- 
fresh Interval decreases as the network call rates In- 
crease, for any given fixed packet-loss error rate. Fur- 



thermore, the refresh interval decreases as the packet- 
loss error rates increase, for any given fixed network call 
rate. 

[001 5] In yet another embodiment of the invention , the 
5 selected refresh interval is an Integer, where the integer 
and factors of the integer are not evenly divisible into 
the integer number of macroblocks per frame. In addi- 
tion, a number of macroblocks between refresh intra- 
coded macroblocks is equal to the selected refresh in- 
to terval minus one. Furthermore, the selected refresh in- 
terval is equal to a number of video signal frames such 
that each macroblock associated with a spatial position 
within the video signal frames is encoded as a refresh 
intra-coded macroblock at least once. 

15 

BRIEF DESCRIPTION OF THE DRAWINGS 
[0016] 

20 FIG. 1 is a table of the resolution and bit rate re- 
quirements for various video fomnats, according to 
the prior art; 

FIG. 2 is a block diagram of an exemplary video 
conferencing system, according to the present in- 
25 vention; 

FIG. 3 is a block diagram of an exemplary video 
conference station of the video conferencing sys- 
tem of FIG. 2; 

FIG. 4 is a block diagram of an exemplary embod- 
30 iment of the image processing engine of FIG. 3; 

FIG. 5 Is a table of intra-coded macroblock refresh 
intervals as a function of call rates and packet-loss 
error rates for a GIF formatted video frame, accord- 
ing to an exemplary embodiment of the invention; 
35 FIG. 6 A Is an exemplary frame comprised of n = 1 2 
macroblocks, according to one embodiment of the 
invention; 

FIG. 6B illustrates five exemplary consecutive 
frames of n = 12 macroblocks per frame with an al- 
40 lowable refresh interval of five; 

FIG. 60 Illustrates three exemplary consecutive 
frames of n = 1 2 macroblocks per frame with a non- 
allowable refresh interval of eight; 
FIG. 7 is an exemplary flowchart of method steps 
45 for generating and transmitting refresh intra-coded 
macroblocks over packet switched networks, ac- 
cording to one embodiment of the Invention; and 
FIG. 8 is an exemplary flowchart of method steps 
for receiving video data over packet switched net- 
so works, according to one embodiment of the inven- 
tion. 

DETAILED DESCRIPTION OF THE DRAWINGS 

55 [0017] The present invention reduces prediction drift 
caused by a mismatch between reference frames on an 
encoder and those on a decoder by dynamically select- 
ing an intra-coded macroblock refresh interval as a func- 
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tion of a packet-loss error rate and a call rate. By spec- 
ifying that each macroblock associated with a spatial ar- 
ea of a video frame be intra-coded at least once within 
the selected refresh interval, one or more lost decoder 
reference frame macroblocks are replaced (i.e., re- 
freshed) by good intra-coded macroblocks within the se- 
lected refresh Interval, thus reducing an occurrence of 
reference frame mismatches. These improvements 
seek to attenuate the disturbances caused by data 
packet loss across a communication link. The scope of 
the present invention covers a variety of video stand- 
ards, including, but not limited to, H.261, H.263, H.264, 
MPEG, MPEG-2, and MPEG-4. 

[0018] FIG. 2 illustrates an exemplary video confer- 
encing system 200. The video conferencing system 200 
includes a local video conference station 202 and a re- 
mote video conference station 204 connected through 
a network 206. Although FIG. 2 only shows two video 
conference stations 202 and 204, those sl<illed in the art 
will recognize that more video conference stations may 
be coupled to the video conferencing system 200. It 
should be noted that the present system and method 
may be utilized in any communication system where vid- 
eo data is transmitted over a network. The network 206 
may be any type of electronic transmission medium, 
such as, but not limited to, POTS, cable, fiber optic, and 
radio transmission media. 

[001 9] FIG. 3 is a block diagram of an exemplary vid- 
eo conference station 300. For simplicity, the video con- 
ference station 300 will be described as the local video 
conference station 202 (FIG. 2), although the remote 
video conference station 204 (FIG. 2) may contain a 
similar configuration. In one embodiment, the video con- 
ference station 300 includes a display device 302, a 
CPU 304, a memory 306, at least one video capture de- 
vice 308, an image processing engine 31 0, and a com- 
munication interface 312. Alternatively, other devices 
may be provided in the video conference station 300, or 
not all above named devices provided. The at least one 
video capture device 308 may be implemented as a 
charge coupled device (CCD) camera, a complementa- 
ry metal oxide semiconductor (CMOS) camera, or any 
other type of image capture device. The at least one vid- 
eo capture device 308 captures images of a user, con- 
ference room, or other scenes, and sends the images 
to the image processing engine 310. When encoded vid- 
eo is sent on a packet switched network (not shown), 
the image processing engine 310 processes the video 
image into data packets before the communication in- 
terface 31 2 transmits the data packets to the remote vid- 
eo conference station 204. The image processing en- 
gine 310 will be discussed in more detail In connection 
with FIG. 4. Conversely, the image processing engine 
31 0 also transforms received data packets from the re- 
mote video conference station 204 into a video signal 
for display on the display device 302. 
[0020] FiG. 4 is an exemplary embodiment of the im- 
age processing engine 310 of FiG. 3. The image 



processing engine 31 0 includes a coding engine 402, a 
macroblock packetization engine 404, and a communi- 
cation buffer 406. In other embodiments of the invention, 
the macroblock packetization engine 404 may be ab- 
5 sent or may be incorporated in the coding engine 402, 
or the image processing engine 310 may include more 
or less elements. 

[0021] Initially, a video signal from the video capture 
device 308 (FIG. 3) enters the coding engine 402, which 

10 converts each frame of video into a desired format, and 
transforms each frame of the video signal into a set of 
macroblocks. A macroblock is a data unit that contains 
blocks of data comprising luminance and chrominance 
components associated with picture elements (also re- 

is ferredtoas pixels). For example, in H.263, a macroblock 
consists of four 8x8 blocks of luminance data and two 
corresponding 8x8 blocks of chrominance data in a 4: 
2:0 chroma sampling fonnat. An 8 x 8 block of data is 
an eight-column by eight-row matrix of data, where each 

20 data corresponds to a pixel of the video frame. A 4:2:0 
chroma formatted macroblock comprises data covering 
a 1 6 pixel by 1 6 pixel section of the video frame. How- 
ever, the present invention is not limited to macroblocks 
as conventionally defined, but may be extended to any 

25 data unit comprising luminance and/or chrominance da- 
ta. In addition , the scope of the present invention covers 
other sampling formats, such as a 4:2:2 chroma sam- 
pling format comprising four 8x8 blocks of luminance 
data and four corresponding 8x8 blocks of chromi- 

30 nance data, or a 4:4:4 chroma sampling format compris- 
ing four 8x8 blocks of luminance data and eight corre- 
sponding 8x8 blocks of chrominance data. 
[0022] In addition, the coding engine 402 encodes (i. 
e.. compresses) each macroblock to reduce the number 

35 of bits used to represent data content. Each macroblock 
may be "intra-coded" or "inter-coded," and a video frame 
may be comprised of any combination of intra-coded 
and inter-coded macroblocks. Inter-coded macroblocks 
are encoded using temporal similarities (i.e., similarities 

40 that exist between a macroblock from one frame and a 
closely matched macroblock from a previously coded 
frame). Specifically, a given inter-coded macroblock 
comprises encoded differences between the given mac- 
roblock and a closely matched macroblock from a pre- 

45 vlous reference video frame. The closely matched mac- 
roblock from the previous reference video frame may 
comprise data associated with pixels that are spatially 
offset from the pixels associated with the given macrob- 
lock. Alternatively, intra-coded macroblocks are encod- 

50 ed without use of infoimation from other video frames. 
[0023] in one embodiment of the invention, the coding 
engine 402 Intra-codes macroblocks of a frame using a 
refresh mechanism. The refresh mechanism is a deter- 
ministic mechanism to eliminate mismatches between 

55 the encoder and decoder reference frames, called pre- 
diction drift, by Intra-coding a specific pattern of mac- 
robloci<s for each frame. For future reference, a mac- 
roblock intra-coded via the refresh mechanism will be 
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referred to as a refresh intra-coded macroblock. The re- 
fresh mechanism will be discussed further below in con- 
junction with FIGS. 5-6. 

[0024] For all the remaining macrobIocl<s of a given 
frame that have not been intra-coded by the refresh 
mechanism, the coding engine 402 encodes each mac- 
roblocl< as intra-coded or inter-coded. For example, to 
determine If a given macrobiocic may be encoded as an 
inter-coded macroblock, the coding engine 402 com- 
putes differences between data of the given macroblock 
of a current video frame with data of a macroblock from 
aprevious (or subsequent as in acase of bi-directionally 
predicted frame coding) reference video frame (referred 
to as an offset macroblock), where the differences may 
be realized, for example, by a mean-absolute error or a 
mean-squared error between data corresponding to pix- 
els located at co-located positions within the macrob- 
locks. For the given macroblock, the coding engine 402 
computes errors for a plurality of spatially offset mac- 
roblocks. If the coding engine 402 only finds errors 
greater than a predetermined difference threshold val- 
ue, then significant similarities do not exist between data 
from the given macroblock and data from the previous 
frame, and the macroblock Is intra-coded. However, if 
an error Is found to be less than the predetemnlned dif- 
ference threshold value for the given macroblock and a 
given offset macroblock from the previous frame, then 
the given macroblock is inter-coded. The scope of the 
present invention covers other types of selection criteria 
to detemnlne whether a given macroblock is intra-coded 
or Inter-coded. 

[0025] To inter-code the given macroblock, the coding 
engine 402 subtracts the given macroblock's data from 
the offset macroblock's data (i.e., luminance and 
chrominance data associated with a pixel of the given 
macroblock is subtracted from luminance and chromi- 
nance data associated with a corresponding pixel of the 
offset macroblock for every pixel) to give difference da- 
ta, encodes the difference data using standard coding 
techniques such as Discrete Cosine Transfonns and 
quantization methods among others, determines an off- 
set vector from the given macroblock to the offset mac- 
roblock (referred to as a motion vector), and encodes 
the motion vector. 

[0026] The coding engine 402 of the Image process- 
ing engine 31 0 (FIG. 3) of the remote video conference 
station 204 (FIG. 2) may use a variety of error conceal- 
ment techniques in conjunction with receiving refresh in- 
tra-coded macroblocks to Improve video quality. For ex- 
ample, in one embodiment of the invention, the coding 
engine 402 decodes the neighboring macroblocks of a 
lost Inter-coded macroblock, estimates a motion vector 
of the lost macroblock, and then uses the estimated mo- 
tion vector to reconstruct data of the lost macroblock. In 
another embodiment of the invention, the coding engine 
402 may decode the neighboring macroblocks of a lost 
intra-coded macroblock, and spatially Interpolate the 
decoded neighboring data to reconstruct the lost data. 



The neighboring macroblocks of the lost inter-coded 
and lost Intra-coded macroblocks may be one or more 
of the received refresh intra-coded macroblocks. The 
scope of the present invention covers other error con- 
5 cealment techniques used in conjunction with refresh in- 
tra-coded macroblocks to improve video quality due to 
lost macroblocks. Alternatively, the image processing 
engine 310 (FIG. 3) of the present Invention may gen- 
erate, transmit, and receive refresh intra-coded macrob- 
10 locks without any other error concealment techniques 
to improve video quality, 

[0027] Once the macroblocks of a given frame are en- 
coded, the coding engine 402 sends the encoded mac- 
roblocks to the macroblock packetization engine 404. 

The macroblock packetization engine 404 places the 
encoded macroblocks into discrete data packets. The 
macroblock packetization engine 404 may place the en- 
coded macroblocks of the given frame into the discrete 
data packets randomly, according to a raster scan order 
of the given frame, according to error concealment tech- 
niques, or according to any other methods. Typically, da- 
ta from more than one encoded macroblock is placed 
Into a given data packet. 

[0028] Subsequently, the data packets are fonvarded 
to the communication buffer 406 for transmission across 
the network 206 (FIG. 2) by the communication interface 
312 (FIG. 3). To further promote resilience against pack- 
et loss, redundant picture headers may be transmitted 
In the data packets. 

[0029] Conversely, the Image processing engine 31 0 
also processes video data packets received from a re- 
mote location and provides video signals for display. In- 
itially, data packets are received by the communication 
interface 312 (FIG. 3), and fonwarded to the communi- 
cation buffer 406. The data packets are then sent to the 
macroblock packetization engine 404, which unpacks 
the macroblocks, and if necessary, orders the macrob- 
locks back into their original, ordered pattern (i.e., pat- 
tern prior to macroblock packetization at the remote vid- 
eo conference station 204 (FIG. 2), which Is typically 
raster-scan). Subsequently, the coding engine 402 func- 
tions as a decoder and reconstructs data of an original 
video frame using the refresh intra-coded macroblocks, 
other intra-coded macroblocks, and inter-coded mac- 
roblocks derived from the original video frame and pre- 
vious video frames, and determines whether a data 
packet was lost In transit across the network 206 (FIG. 
2). 

[0030] For example, the refresh intra-coded macrob- 
locks are used in lieu of motion compensation from ref- 
erence frames that may be corrupted due to video data 
transmission errors. Any macroblock that may be cor- 
rupted due to video data transmission errors (and not 
corrected by an intra-coded macroblock) further propa- 
gates and typically magnifies prediction drift when the 
coding engine 402 uses the corrupted macroblocks as 
reference macroblocks for decoding other received in- 
ter-coded macroblocks. Thus, the refresh intra-coded 
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macroblocks provide the coding engine 402 with a 
"fresh" set of intra-coded macroblocks to be used as ref- 
erence macroblocks for subsequent decoded frames, 
thereby reducing prediction drift. 
[0031] Typically, the coding engine 402 uses a variety 
of error concealment techniques to rebuild lost macrob- 
locks employing such methods as spatial interpolation 
or motion vector estimation. These error concealment 
techniques are enhanced by using refresh intra-coded 
macroblocks. It should further be noted that although the 
same components are described herein as being used 
for both transmission and receiving functions, the com- 
ponents may be embodied In separate receiver and 
transmitter devices. 

[0032] In another embodiment of the present inven- 
tion, the coding engine 402 may generate an Intra-mac- 
roblock map that identifies which macroblocks in a cod- 
ed video frame are intra-coded. After the intra-macrob- 
lock map is generated, the Image processing engine 
31 0 sends the map to the remote video conference sta- 
tion 204 (FIG. 2). The map may be sent as part of a pic- 
ture header data associated with the coded video frame, 
for example, although other data fields may be used. 
[0033] FIG. 5 Is a table of Intra-coded macrobiock re- 
fresh intervals as a function of calf rates and packet-loss 
error rates for a GIF formatted video frame, according 
to an exemplary embodiment of the invention. The re- 
fresh intervals are given in number of frames. The FIG. 
5 refresh intervals are for illustrative purposes, and are 
not meant to be exclusive of other values. In addition, 
the present Invention covers other refresh intervals and 
other video formats. 

[0034] In one embodiment of the present invention, 
the coding engine 402 (FIG. 4) selects a refresh interval 
based upon a current network call rate and a current 
packet-loss error rate, and Intra-codes a pattern of mac- 
roblocks as dictated by the selected refresh interval. The 
selected refresh interval is equivalent to a number of 
frames processed by the coding engine 402 such that 
every macrobiock associated with a spatial position 
within a frame is intra-coded at least once after the 
number of frames specified by the refresh Interval are 
processed. Since an intra-coded macrobiock typically 
requires a larger number of bits than an inter-coded 
macrobiock, the present invention optimizes the number 
of refresh intra-coded macroblocks based upon network 
communication parameters such as call rates and pack- 
et-loss error rates. As discussed further below in con- 
junction with FIGS. 6A-6C, for any given selected re- 
fresh Interval (ri), a number of macroblocks between re- 
fresh intra-coded macroblocks Is equal to ri - 1 . There- 
fore, for example, If ri = 5, then the coding engine intra- 
codes every fifth macrobiock (i.e., there are four mac- 
roblocks between refresh intra-coded macroblocks, 
since ri - 1 = 4) such that every macrobiock associated 
with a spatial position within a frame is intra-coded at 
least once after five frames are processed as long as 
the number of macroblocks in the frame is not a multiple 



of five. 

[0035] The FIG. 5 embodiment of the present inven- 
tion specifies a refresh interval (In number of frames) 
that decreases as the data rate of the video call (here- 

5 aftercalled the call rate) increases, for any given packet- 
loss error rate. For example, when less than 3% of the 
packets transmitted across the network 206 (FIG. 2) are 
lost (i.e., the packet-loss error rate is less than 3%; col- 
umn 2, FIG. 5), the refresh Interval decreases from 149 

10 frames when the call rate is less than or equal to 384 
kbps to 65 frames when the call rate is greater than or 
equal to 768 kbps. That Is, as the call rate increases (for 
a given packet-loss error rate), more bandwidth is avail- 
able to accommodate a larger number of refresh intra- 

15 coded macroblocks and still maintain good video quality. 
In addition, the FIG. 5 embodiment of the present inven- 
tion specifies a refresh interval that decreases as the 
packet-toss error rate Increases, for any given call rate. 
For example, when the call rate is greater than or equal 

20 to 768 kbps (row 4, FIG. 5), the refresh Interval decreas- 
es from 65 frames when the error rate is less than or 
equal to 3% to 29 frames when the error rate is greater 
than 10%. That is, as more packets are lost, the proba- 
bility of prediction drift increases, and therefore the cod- 

25 ing engine 402 Intra-codes a higher percentage of re- 
fresh macroblocks per frame to eliminate the persist- 
ence and propagation of errors due to packet loss. 
[0036] FIG. 6 A is an exemplary frame 600 comprised 
of twelve macroblocks, according to one embodiment of 

30 the invention. For Illustrative purposes, the twelve mac- 
roblocks are labeled 1 - 12. in one embodiment of the 
present invention, a refresh interval is any integer that 
has the following properties: a given integer divided into 
the number of macroblocks per frame (n) does not give 

35 an Integer quotient (I.e., the given integer is not evenly 
divisible Into the number of macroblocks per frame (n)), 
and the given integer does not have any factors that 
when divided Into the number of macroblocks per frame 
(n) give Integer quotients (i.e., any factors of the given 

40 integer are not evenly divisible Into the number of mac- 
roblocks perframe(n)). This property ensures that every 
macrobiock location will be intra-refreshed after the cod- 
ing engine 402 processes ri frames, where ri Is the re- 
fresh interval. Thus, for n = 12, the refresh interval may 

45 be five, seven, or eleven, since five, seven, and eleven 
are not evenly divisible Into n = 1 2, and five, seven, and 
eleven do not have factors that are evenly divisible Into 
n = 12. However, the refresh interval cannot be any in- 
teger from the set of integers {1, 2, 3, 4, 6, 12} in the 

50 present embodiment, since any integer from the set of 
integers {1 , 2, 3, 4, 6, 12} is evenly divisible into n = 12. 
Furthermore, the refresh interval cannot be eight, since 
eight can be factored into four and two, both of which 
are evenly divisible into n =1 2. Likewise, the refresh in- 

55 terval cannot be nine, since nine can be factored into 
three and three, and three Is evenly divisible into n = 12. 
Finally, the refresh rate cannot be ten, since ten can be 
factored into five and two, and two Is evenly divisible Into 
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n = 12. 

[0037] FIG. 6B illustrates five consecutive frames 602 
of n = 12 macroblocks per frame witli an allowable re- 
fresh Interval of five, according to one exemplary em- 
bodiment. For illustrative purposes, the macroblocks of 
frames 602 are arranged in a linear sequence. After the 
coding engine 402 (FIG. 4) selects an allowable refresh 
Interval of five (I.e., rl = 5), the coding engine 402 then 
Intra-codes every fifth macroblock as a refresh Intra- 
coded macroblock. In other words, there are four mac- 
roblocks between refresh intra-coded macroblocks, 
since ri - 1 = 4. Forexample. the coding engine402 intra- 
codes a first macroblock 604 of a first frame 602a, skips 
four macroblocks and intra-codes a sixth macroblock 
606 of the first frame 602a, skips four macroblocks and 
intra-codes the eleventh macroblock 608 of the first 
frame 602a, and skips four macroblocks and intra-codes 
the fourth macroblock 61 0 of a second frame 602b. The 
coding engine 402 continues to sequentially intra-code 
every fifth macroblock of the remaining macroblocks In 
frames 602b-602e, as well as every fifth macroblock in 
subsequent frames (not shown) . Afterthe coding engine 
402 intra-codes the macroblocks of five consecutive 
frames, such as frames 602a-602e, for example, each 
macroblock of macroblocks 1-12 are Intra-coded once. 
Therefore, when the coding engine 402 selects a refresh 
interval ri = 5, the coding engine 402 intra-codes every 
fifth macroblock of consecutive frames, and intra-codes 
every macroblock associated with a spatial position 
within any frame once after processing the five consec- 
utive frames 602. 

[0038] FIG. 6C illustrates three consecutive frames 
612 of n = 12 macroblocks per frame with a non-allow- 
able refresh interval of eight. According to the present 
invention, a refresh interval of eight (i.e., ri = B) is not 
allowable. The reason rl 8 is because the coding en- 
gine 402 skips seven (i.e., rl - 1 = 7) macroblocks be- 
tween refresh intra-coded macroblocks, and conse- 
quently not every macroblock is intra-coded at least 
once. For example, the coding engine 402 intra-codes 
a first macroblock 614 and a ninth macroblock 61 6 of a 
first frame 61 2a, and a fifth macroblock 61 8 of a second 
frame 61 2b. However, the intra-coding pattern of the first 
frame 612a would be repeated beginning with a third 
frame 612c, since the coding engine 402 intra-codes a 
first macroblock 620 and a ninth macroblock 622 of the 
third frame 612c, which occupy identical spatial posi- 
tions within the third frame 61 2c as the first intra-coded 
macroblock 614 and the ninth intra-coded macroblock 
616 within the first frame 612a. Therefore, a refresh in- 
terval of eight does not allow macroblocks 1 -1 2 to be 
intra-coded at least once, since only macroblocks one, 
five, and nine are intra-coded, independent upon the 
number of frames processed by the coding engine 402. 
[0039] FIG. 7 is an exemplary flowchart 700 of method 
steps for generating and transmitting refresh Intra-cod- 
ed macroblocks over packet switched networks, accord- 
ing to one embodiment of the present invention. In step 



705, at least one video capture device 308 (FIG, 3) cap- 
tures a video image and generates a video signal. 
[0040] Next, in step 71 0, the coding engine 402 (FIG. 
4) (also referred to as an encoder when processing data 

5 for transmission) receives the video signal and trans- 
forms each frame of the video signal into a set of mac- 
roblocks. In step 715, the coding engine 402 selects a 
refresh Interval from a set of predefined Integers based 
upon a call rate and a packet-based error rate, and intra- 

10 codes a pattern of refresh macroblocks based upon the 
selected refresh interval. In one embodiment of the in- 
vention, the set of predefined integers are stored In the 
memory 306 (FIG. 3). In an alternate embodiment of the 
invention, the coding engine 402 computes the refresh 

15 interval based upon the call rate and the packet-based 
error rate. Given a video format which specifies a given 
number of macroblocks per frame, the set of predefined 
integers Includes all Integers except those Integers that 
when divided Into the given number of macroblocks per 

20 frame give integer quotients (i .e. , except those integers 
that are evenly divisible into the given number of mac- 
roblocks per frame), and except those integers com- 
prised of one or more factors that when divided into the 
given number of macroblocks per frame give integer 

25 quotients (i.e., except those integers comprised of one 
or more factors that are evenly divisible into the given 
number of macroblocks per frame). The coding engine 
402 then encodes the remaining macroblocks of each 
frame, based upon selection criteria that determines 

30 whether a given macroblock is intra-coded or inter-cod- 
ed. A video frame may comprise inter-coded macrob- 
locks. intra-coded macroblocks, or any combination of 
intra-coded and inter-coded macroblocks. 
[0041] Next, the macroblock packetization engine 

35 404 (FIG. 4) creates discrete data packets and places 
the encoded macroblocks into the discrete data packets 
in step 720. The encoded macroblocks may be placed 
into the discrete packets in a raster scan order of the 
macroblocks or some other predetermined order in the 

40 video frame, or according to error concealment tech- 
niques disclosed in Patent Application Serial No. 
1 0/226,504, filed August 23, 2002, entitled "System and 
Method for Video Error Concealment," which is incorpo- 
rated herein by reference. However, the scope of the 

45 present invention includes any method of grouping and 
packetlzing the encoded macroblocks for transmission 
over the network 206 (FIG. 2). Alternatively, the mac- 
roblock packetization engine 404 may be a transport en- 
gine for placing the encoded macroblocks into a partic- 

50 ular fomnat for transport on a circuit-switched network 
(not shown). Finally, In step 725, the discrete data pack- 
ets are sent to the communication buffer 406 (FIG. 4) 
for transmission to the remote video conference station 
204 (FIG. 2) via the network 206. 

55 [0042] FIG. 8 is an exemplary flowchart 800 of method 
steps for receiving video data over packet switched net- 
works, according to the present invention. In step 805, 
the communication buffer 406 (FIG. 4) receives trans- 
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mitted data packets from the remote video conference 
station 204 (FIG. 2) via tlie networl< 206 (FIG. 2). Then, 
in step 810, the macroblocic pacl<etization engine 404 
(FIG. 4) de-packetizes the received data packets into 
encoded macroblocks. If necessary, the macroblock 
packetlzation engine 404 also orders the encoded mac- 
roblocks and places the encoded macroblocks in proper 
spatial configuration within a video frame. 
[0043] Next, the coding engine 402 (FIG. 4) decodes 
the encoded macroblocks in step 815. For example, if 
a given macroblock is an intra-coded macroblock, then 
the coding engine 402 decodes the intra-coded macrob- 
lock and places it in a newly decoded video frame. In 
addition, the coding engine 402 decodes inter-coded 
macroblocks to generate decoded residual, adds the 
decoded residual to data content of reference frame 
macroblocks, and places the results in the newly decod- 
ed video frame. The given macroblock may or may not 
be a refresh intra-coded macrobtock. 
[0044] Optionally, in step 820, the coding engine 402 
(functioning as a decoder) or some other mechanism 
related to a video data packet transform (e.g., RTP se- 
quence numbers) determines which macroblocks, if 
any, are lost In transit, and reconstructs these lost mac- 
roblocks using one or more data reconstruction 
schemes. Data reconstruction schemes such as motion 
estimation/compensation and data interpolation are de- 
scribed in the Patent Application Serial No. 10/226,504 
entitled "System and Method for Video Error Conceal- 
ment." Finally, once the data contents of any missing 
macroblocks have been reconstructed, the macrob- 
locks are displayed by the display device 302 (FIG. 3) 
in step 825. 

[0045] The invention has been explained above with 
reference to exemplary embodiments. It will be evident 
to those skilled In the art that various modifications may 
be made thereto without departing from the broader 
spirit and scope of the invention. Further, although the 
invention has been described in the context of Its imple- 
mentation in particular environments and for particular 
applications, those skilled in the art will recognize that 
the present invention's usefulness is not limited thereto 
and that the invention can be beneficially utilized in any 
number of environments and implementations. The 
foregoing description and drawings are, accordingly, to 
be regarded in an illustrative rather than a restrictive 
sense. 



Claims 

1 . A method for video error concealment of a video sig- 
nal, comprising the steps of: 

receiving a plurality of macroblocks, each 
frame of the video signal having an Integer 
number of macroblocks; 
selecting a refresh interval based upon one or 



more of network call rates and packet-loss error 
rates; and 

encoding a subset of the plurality of macrob- 
locks as refresh Intra-coded macroblocks 
5 based upon the selected refresh Interval. 

2. The method of claim 1 , wherein the selected refresh 
interval is an integer m, the Integer m and factors of 
the integer m when divided Into the Integer number 

10 of macroblocks per frame do not give integer quo- 
tients, and wherein the step of encoding a subset 
of the plurality of macroblocks further comprises in- 
tra-codlng every m^ macroblock as a refresh Intra- 
coded macroblock. 

15 

3. The method of claim 1 , further comprising the step 
of placing the encoded macroblocks into discrete 
data packets for transmission across a packet- 
switched network. 

20 

4. The method of claim 1 , further comprising the step 
of placing the encoded macroblocks into a particu- 
lar fomnat for transport on a circuit-switched net- 
work. 

25 

5. A system for video error concealment, comprising: 

. a coding engine configured to 

30 process each frame of a video signal to 

generate macroblocks, each frame having 
an integer number of macroblocks depend- 
ent upon a video signal format, 
select a refresh interval based upon one or 
35 more network communication parameters, 

and 

encode a subset of macroblocks from the 
generated macroblocks as refresh intra- 
coded macroblocks based upon the select- 
^0 ed refresh interval; and 

a transport engine configured to place each of 
the encoded macroblocks into a particular for- 
mat for transmission across a network. 

45 

6. The system of claim 5, wherein the selected refresh 
Interval is an integer m, the integer m and factors of 
the integer m when divided into the integer number 
of macroblocks per frame do not give integer quo- 

50 tients, and wherein the coding engine encodes the 
subset of macroblocks by intra-coding every m^^ 
macroblock of the generated macroblocks as a re- 
fresh intra-coded macroblock. 

55 7, The system of claim 5, wherein the transport engine 
places each of the encoded macroblocks Into a par- 
ticular format for transport on a circuit-switched net- 
work. 
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The system of claim 5, wherein the transport engine 
places each of the encoded macroblocks into pack- 
ets for transmission across a packet-switched net- 
work. 
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